Advanced user interface

ABSTRACT

An advanced user interface for use with a computer system operating on an integrated operating environment. The integrated operating environment allows a plurality of application programs to be running simultaneously, one of which is designated the active application program to which all input data is directed. The advanced user interface allows a user to select among user-friendly input devices to operate any application program according to his individual preferences without change to the application program code. The advanced user interface includes alternate input modules which translate the input signals transmitted from the various input devices into input messages useable by the rest of the interface. The advanced user interface also includes interface profiles which contain mappings of the input messages against corresponding commands useable by the application programs, the integrated operating environment or other modules of the advanced user interface itself. An environment link module refers to the interface profiles and matches the input message against the corresponding command for the application program active at the time the input signal was transmitted and send the corresponding command to that application program.

BACKGROUND OF THE INVENTION

This invention relates to improvements in entering input data intodigital computers. More particularly, it relates to an advanced userinterface which allows a user to select one or more input devices toinput data into a computer running a program originally written for adifferent input device in a multiapplication environment.

In the past, computers were used only by scientists, mathematicians, andother high-level, sophisticated computer users. As computer technologyprogressed, and particularly with the advent of the personal computer,data processing has reached every level of society, and every level ofuser. The trend is for fewer computer users to be computer professionalsor sophisticated in data processing techniques. Access to computers willincrease even further in the future as computer hardware and softwareincrease in power and efficiency.

However, despite the prevalence of computers in our society, many areapprehensive about using them. The traditional text display andalphanumeric keyboard input device are not readily learned, requiringthe memorization of the correct format of dozens, if not hundreds, ofcommands. Unless the user is a reasonably good typist, data entry can beinconvenient and time consuming. The hours of experimentation which maybe necessary to become proficient with a single piece of softwarebecomes extremely frustrating for the unsophisticated user who is likelyto avoid the effort altogether.

It has therefore become necessary to design what have become known inthe art as "user friendly" input devices and computer programs. Such"user friendly" devices and software are designed to allow anunsophisticated user to perform desired tasks without extensivetraining. One of the more popular input devices is the mouse pointingdevice. The mouse generates signals corresponding to the direction anddistance it is moved across a flat surface by the user. The computerresponds to this information by changing the position of the cursor onthe computer display to allow the user to point to any displayed object.Once the cursor is correctly positioned, the user can perform a numberof functions by depressing one or more buttons on top of the mouse.

Human factor studies have shown that a device which allows the user toinput data directly on the visual display screen of a computer,generally known in the art as a touch input device, achieves greatestimmediacy and accuracy between man and machine. One of the first inputdevices for use at the display surface was the light pen. The light penis an optical detector in a hand held stylus, which is placed againstthe face of a cathode ray tube. The location of the light pen isdetermined by detecting the coordinates of the dot of light which is thescanning raster of the display. A second type of touch input device is amechanical deformation membrane which is placed over the display screen.The membrane is a transparent overlay which consists of two transparentconductor planes disposed on a flexible surface. When a selection ismade, the user mechanically displaces one of the conductor planes totouch the other by a finger or stylus touch, thereby bringing theconductors into electrical contact with each other. Appropriateelectronics and software translate the electrical signals generated bythe finger or stylus touch to the position on the display surface.Another touch input device is a capacitive transparent overlay placedover the display screen, which includes transparent conductors driven byan electromagnetic signal. The input device can detect the location of afinger touch by the change in capacitance of the overlay or,alternately, a stylus is used to return the electromagnetic signals fromthe overlay back to the computer to determine the stylus position. Yetanother touch input device uses a frame which fits around the displayscreen having a number of infrared or visible light transmitters andreceptors arranged in parallel horizontal and vertical directions. Whenthe user's finger blocks the light beams, the horizontal and verticalreceptors note the absence of the signals, thereby locating the positionof the action desired by the user.

Many other user friendly input devices which respond to actions of theuser such as voice recognition units or digital tablets, have beendeveloped.

In addition to user friendly input devices, designers have made effortsto develop more user friendly software. One technique which has beenemployed is to provide the user with a menu of choices of the particulartasks or functions which can be performed. In this way, the user is notrequired to commit long lists of commands to memory. The menu can be afull or partial screen display with spaces adjacent to the menu entriesto which the cursor is moved by keyboard or by other cursor movingdevice to select a particular action. Alternatively, the user can selectan action by entering an alphanumeric character associated with the menuselection on a command line.

Another recent trend is to provide some sort of integration of computerprogram applications. Without integration, the user must employ separateapplication programs for word processing, database manipulation,graphics and electronic mail functions, and so forth. It is often quitedifficult to integrate the outputs of the different programs into asingle desired output. One solution has been to write a singleintegrated piece of software which incorporates a variety ofapplications which is called a multiple-function program. Typically,these multiple-function programs include text, spreadsheet, and businessgraphing applications.

Another approach is to provide an integrated operating environment asimplemented by Microsoft WINDOWS™ or IBM Presentation Manager™. In thisapproach, individual application programs share information and oftenappear on the display screen at the same time, each in its own window.By selecting the strongest individual application programs, a much morepowerful environment can be tailored for the user's particular needs, incontrast to multi-function programs where the user is limited towhatever applications have been programmed into the package.

Unfortunately, while many user friendly input devices and many userfriendly computer programs have been developed, these efforts have notbeen well integrated. For example, there are a large number of programswhich have been written to accept keyboard or mouse input which do notrecognize information from a touch input device or voice recognitionunit. This situation presents a severe obstacle to the introduction andwidespread use of new user friendly input devices. Either a programmermust make extensive revisions to a large body of existing software, ormust write a new multiple function program, or set of applicationprograms each time a new input device is developed. Either alternativeadds significantly to the costs and time of a development effort and maybe beyond the capabilities of the organization developing the inputdevice or render the effort too expensive for the expected benefits. Inaddition, copyrights or other proprietary rights may prevent the inputdevice developer from adapting existing software. Further, newlydeveloped software may not be as efficient or commercially acceptable tousers as the established application programs already written for aparticular purpose.

A final consideration is that the individual user is probably bestsuited to assessing his own needs, yet is least likely to be able orinterested in undertaking extensive revisions in applicationprogramming. It would be advantageous to allow a user to select theparticular input devices and application programs which would best servehis purposes and yet with only a relatively low level of expertise,allow him to assemble the components in a user friendly interfacebetween himself and the computer. The general solution developed by theprogrammer may not be optimum for a particular user's needs. It wouldalso be advantageous to assure a user that future advances in inputdevices and application programs can be accommodated by the interfacewith a minimum of revision.

SUMMARY OF THE INVENTION

It is therefore an object of the invention to allow a user to selectbetween a plurality of input devices to input data into a computersystem.

It is another object of the invention to input information to a computersystem via a different input device than that for which the activeapplication program was originally written without revising theapplication program code.

It is still another object of the invention to allow a relativelyunsophisticated user to select among available application programs andinput devices and construct a user interface designed particularly tohis needs.

It is yet another object of the invention to accommodate future advancesin user friendly input devices and application programs in a userinterface.

It is still yet another object of the invention to allow different userswho operate a single computer to construct different user interfaces toaccommodate individual preferences in inputting data.

These objects and others are accomplished by an advanced user interfacewhich operates with an integrated operating environment capable ofrunning a plurality of application programs and which utilizes themessage and focusing functions provided by the environment. The advanceduser interface is comprised of three major modules, an alternative inputsubsystem module, an interface profile module, and an environment linkmodule. The alternative input subsystem module provides communicationbetween the attached user friendly input devices and the remainder ofthe advanced user interface as well as application programs through theintegrated operating environment. The alternative input subsystem modulereceives the input signals generated by the input devices and translatesthem to input messages useable in the advanced user interface.

The interface profile module comprises a set of application profiles,one for each application program for which the user wishes to use aninput device in a different manner from the way in which the applicationprogram was originally written. An entirely new input device may be usedwith the application program, or optionally, the user may change theinput signals of the existing input device from those to which theapplication program originally responded. Each application profilecontains a list of input messages mapped to corresponding commandsuseable by the application program. The interface profile module mayalso comprise a set of user profiles for each user who utilizes thecomputer system. The user profile contains a list of input messages andtheir corresponding commands which can be common to several applicationprograms.

The environment link module communicates with integrated operatingenvironment and contains algorithms for the execution of the advanceduser interface procedures. For example, it takes the input messagesgenerated by the alternate input subsystem, queries the integratedoperating environment as to which application program is active, matchesthe input messages to the corresponding commands in the appropriateapplication profile within the interface profile module, and initiatesan action based on the set of instructions such as sending messages tothe active application program via the operating environment. Theadvanced user interface may also include a set of utility programs suchas a pop-up keyboard or an image magnifier utility.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the interaction of computer systemsoftware and hardware in response to actions of user using an integratedoperating environment such as Presentation Manager™.

FIG. 2 is an architectural block diagram of a computer system inaccordance with the present invention, depicting the memory having readtherein a series of application and operating system programs.

FIG. 3 is a block diagram showing the interaction of computer systemsoftware and hardware in response to actions of a user designed inaccordance with a first embodiment of the present invention.

FIG. 4 is a flow diagram of the operation of the first embodiment of thepresent invention in response to a voice input signal.

FIG. 5 is a diagram of an application profile.

FIG. 6 is a block diagram showing the interaction of computer systemsoftware and hardware in response to actions of a user designed inaccordance with a second embodiment of the present invention.

FIG. 7 is a flow diagram of the operation of the second embodiment ofthe present invention in response to a voice input signal.

FIG. 8 is a flow diagram of the operation of a computer module whichdistinguishes between touch input meant to emulate a mouse pointingdevice and touch input meant to be a gesture or character.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 1 shows the interaction of the user and the computer systemutilizing application programs A, B, C and D 50-53 in a typicalintegrated operating environment such as Presentation Manager™ 56.

The user inputs data into the computer system via device A (keyboard)32, device B (mouse) 34 or touch sensor 36. As the user inputs data, theselected device generates an interrupt to activate the appropriatedriver. In the case of the keyboard 32, driver A 60 is activated; in thecase of the mouse 34, driver B 62 is energized; and input to the touchsensor 36 activates the touch driver 63. The appropriate driver signalsthe integrated operating environment 56. The integrated operatingenvironment 56 keeps track of which application program is active andsends all input data to that application program. Alternatively, theuser can change application programs, and the integrated operatingenvironment 56 will route the message to the new active applicationprogram. In one preferred embodiment, application program A 50 is aspreadsheet program, application program B 51 is a word processor,application program C 52 is a drawing program, and application program D53 is an electronic mail facility. Keeping track of the active programis a "focusing function", as the integrated operating environment 56determines the "focus" of the user at a given time. For purposes ofillustration, assume the user is inputting data via the keyboard 32 andthe spreadsheet 50 is the active application program. The spreadsheet 50then decides what action should be performed. Generally, this actionwill fall into one of three categories: the spreadsheet 50 will changethe display 28, the spreadsheet 50 will communicate with other I/Odevices such as a printer, or the spreadsheet 50 will communicate withother applications. All of the messages are handled by the integratedoperating environment 56.

While the prior art integrated operating environments allow easieraccess to multiple application programs, and therefore, the user canselect the particular application programs which best suit his needs,the user can only communicate with the application program with theinput devices for which it has been written. For example, spreadsheet 50is a program which has been written for a combination of keyboard 32 andmouse 34. Although a touch driver 63 can be developed to communicatecoherently with the integrated operating environment 56, by convertinginput signals generated by the touch sensor 63 into mouse movements andmouse clicks, the capabilities of the touch input device 36 are limitedby the repertoire of the mouse device. New categories of touch commandssuch as handwriting recognition or gestures cannot be easilyaccommodated by this method, especially by an application program suchas spreadsheet 50 which has no understanding of what the types of touchcommands mean. Other new input devices are more drastically limited.Thus, although the system designer or user would like to include newuser-friendly input devices into an individual workstation, he may findit impossible or severely limiting to use existing software.

The preferred embodiment of the invention comprises a set of computerprograms for controlling the interaction between a user and a computersystem as shown in FIG. 2. The invention is primarily envisioned for usewith a personal computer such as the IBM PS/2™; however, the principlesof this invention can be extended to other individual workstations or tomuch larger data processing systems. The architectural block diagram ofFIG. 2 includes a central processing unit (CPU) 20 connected by means ofa system bus 22 to a read-only memory (ROM) 24 and a random accessmemory (RAM) 26. Also included in the computer system in FIG. 2 are adisplay 28 by which the computer presents information to the user, andplurality of input devices including a keyboard 30, and devices A 32 andB 34, which for the purposes of illustration in the followingdescription are a second alphanumeric keyboard 32 and a mouse 34, butwhich could be a specialized keyboard such as a numeric keyboard, and aremote pointing device or a variety of other input devices. The computersystem further comprises a touch sensor 36 or touch input device for useat the surface of the display 28, a voice sensor 38 and an image sensor40. The I/O 42 can be connected to communication lines or other I/Oequipment.

The random access memory (RAM) 26 includes application program A 50,application program B 51, application program C 42, and applicationprogram D 53. Examples of typical application programs would includeword processors, spreadsheets, graphic programs, electronic mail, anddata base programs. The RAM 26 also stores an operating system 54, suchas DOS™ or OS/2™, and an integrated operating environment program 56,such as WINDOWS™ or Presentation Manager™, which allow severalapplication programs to be running concurrently. Other software includesdriver A 60 and driver B 62, which interpret the electrical signalsgenerated by the second keyboard 32 and the mouse 34 respectively forthe operating environment 56. The touch driver 63, the gesturerecognition unit 64 and character recognition unit 65 handle inputsignals from the touch sensor 36. Input signals from the voice sensor 38and the image sensor 40 are processed by the voice recognition unit 66and image recognition unit 68 respectively. Although depicted in FIG. 2as entirely software, driver A 60, driver B 62, touch driver 63, gesturerecognition unit 64, voice recognition unit 66 and image recognitionunit 68 can also be a combination of hardware and software. Finally,also resident in RAM 26 are the set of software modules which make upthe advanced user interface 100.

First Embodiment

FIG. 3 shows a computer system designed in accordance with the presentinvention. Rather than being connected to the integrated operatingenvironment 56 via a standard device driver at least one of theplurality input devices is connected to the advanced user interface 100,via a recognition unit. In FIG. 3, the voice sensor 38 and image sensor40 are connected to the advanced user interface (AUI) 100 via theirrespective recognition units, voice recognition unit 65 and imagerecognition unit 68. A keyboard 32, and a mouse 34, remain connected tointegrated operating environment 56 via their respective drivers driverA 60 and driver B 62, since many application programs recognize keyboardand mouse input. Finally, in FIG. 3, the touch sensor 36 is connected toboth the integrated operating environment 56 via the standard touchdriver 63 and to the advanced user interface 100 via the gesturerecognition unit 64 and the character recognition unit 65. The touchdriver 63, gesture recognition unit 64 and character recognition unit 65translate signals from the touch sensor 36 to produce gesture orcharacter "tokens" respectively. The series of positions of a finger orstylus on the surface of the touch sensor 36 can be recognized ashandwritten alphanumeric characters by the character recognition unit65, or as a gesture such as a circle or right hand arrow by the gesturerecognition unit 64. The voice recognition unit 66 digitizes the soundreceived by the voice sensor 38 and performs pattern matching againstpreviously defined sound patterns to produce voice "tokens". Imagerecognition unit 68 processes information from the image sensor 40 andoutputs image "tokens". A "token" is an input message which represents aseries of electrical signals collected from an input device. Forexample, the series of points collected when a user draws a circle onthe touch sensor can be defined as the "circle token". Coupling thetouch sensor 36 to integrated operating environment 56 allows the userto take advantage of application programs which are specifically writtenfor the touch sensor 36 or other existing application programs for whichonly primitive mouse input signals are needed. Yet as the touch sensor36 is also connected to AUI 100 through the gesture recognition unit 64and the character recognition unit 65, application programs which do notrecognize touch input can still accept gesture or character "tokens" dueto the translation capabilities of the advanced user interface 100.

In other embodiments, the keyboard 32 and mouse 34 are connected to boththe AUI 100 and integrated operating environment 56 as is the touchsensor 36. In still other embodiments, all input devices are attacheddirectly to the AUI 100 alone. By using a keyboard AUI link, a usercould change the normal typewritten commands for an application programto those which he finds more convenient to input or memorize. As the AUI100 is written as an application program, it is able to use thecommunication and focusing capabilities of the integrated operatingenvironment.

Another preferred embodiment of the present invention uses a singlepointing device driver (not pictured) to combine the function of driverB 62 and touch driver 63 by accepting input from both mouse 34 and touchsensor 36 as well as any other pointing device. This embodiment isparticularly adapted to operate with the touch input device described inU.S. Pat. No. 4,686,332, to E. Greanias, et al., entitled "CombinedFinger Touch and Stylus Detection System for Use on the Viewing Surfaceof a Visual Display Device", filed Jun. 26, 1986, or the remotely sensedstylus described in commonly assigned patent application, Ser. No.264,409, filed Oct. 28, 1988, entitled "Remotely Sensed PersonalStylus", by E. Greanias, et al., which are hereby incorporated byreference. The pointing device driver arbitrates between mouse inputfrom the mouse 34 and finger and stylus input from the touch inputdevice 36 and sends appropriate input messages to the integratedoperating environment 56.

Referring back to FIG. 3, the AUI 100 is further divided into severalcomputer code modules. The environment link module 101 containsalgorithms for the execution of the procedures of the AUI 100 andprovides communication capabilities to the integrated operatingenvironment 56. The Alternative Input Subsystem (AIS) 103 provides aninterface description which allows the designers of new user friendlyinput devices, e.g., voice sensor 38, to seamlessly interconnect withthe Advanced User Interface 100. The AIS 103 translates the inputsignals sent by the input devices and their respective recognition unitsto input messages useable by the remainder of the Advanced UserInterface 100. A part of each input message, such as a prefix GS/ todenote a gesture, will be devoted to identify the input device fromwhich the input signal is received. In this embodiment, the AIS 103 alsotime stamps each input message so that the correct application programreceives the appropriate commands.

The interface profile module 104 is comprised of sets of applicationprofiles 105 and the user profiles 107, which are files which list inputmessages produced by the AIS 103 from the input signals received byinput devices 36, 38, 40, mapped to keyboard, mouse or other commandswhich are usable by existing application programs, e.g., mouse clicks,keystroke messages, MACROs, utility programs, etc. The Advanced UserInterface utility (AUI utility) 109 is a set of utility programs such asa pop-up keyboard or an image magnifier utility which can be used withAUI 100. As shown, the AUI utility 109 can be written as a separateapplication program, or set of application programs, so that it can usethe message passing capabilities of the integrated operating environment56 to communicate with other application programs.

A few examples of utilities which might be found in the advanced userinterface utilities 109 include a "pop-up" keyboard, a "pop-up" numerickeypad, an image magnifying utility and an electronic mail routing slip.The "pop-up" keyboard and "pop-up" keypad are functions which arepreferably designed to operate with a touch input device at the displaysurface. By issuing the proper command, e.g., a circle gesture ortouching a keyboard icon, the keyboard or keypad will "pop-up" on thescreen. A user can quickly enter columns of alphanumeric informationdirectly on the screen by touching the keys displayed on the screen,thus eliminating the need to divert their attention to manual keyboard.The space needed for the keyboard or keypad is quite small, and will notcompletely obscure the work area of the spreadsheet, word processor,etc., which is the active application program. The image magnifyingutility will magnify a rectangle of fixed size around a point at whichthe appropriate gesture was made. The utility allows very accuratepositioning of a cursor in the expanded image. After stylus liftoff, thenormal size display is restored, and the selected cursor coordinates aresent to the active application program.

The flow diagram in FIG. 4 depicts the operation of the computer systemin FIG. 3 in response to a voice input signal. The active applicationprogram is the spreadsheet 50, and presented to the user on the display28 is an array of selected data concurrently with a menu bar of possiblefunctions available to the user. Rather than striking a series of keyson the keyboard, the user elects to input data into the computer systemusing another input device, the voice sensor 38. In response to the uservocalizing the word "GRAPH", at 111 the voice sensor 38 generateselectrical signals and sends them to the appropriate translation device,the voice recognition unit 66. In response to these electrical signals,the voice recognition unit is activated at 113 and sends an appropriateinput signal, a voice input "token", to the Alternate Input Subsystem103. At 115, the AIS 103 forwards an input message to the environmentlink 101 which contains information on the time the input signal wasreceived and which input device sent the signal. In response to theinput message, at 117, the environment link 101 queries the integratedoperating environment 56 as to which application program was active atthe time the input signal was received. After determining the activeapplication program, spreadsheet 50 at 119, the environment link 101refers to the application profile 105 of the spreadsheet 50 for thecommand which corresponds to the input message "GRAPH" which will berecognized by the spreadsheet 50, i.e., the menu selection-- graph--.The environment link 101 then at 121 checks the user profile 107 todetermine whether there is a higher priority command which wouldoverride the command from the application profile 105. The environmentlink 101 determines which command has higher priority at 123. Theenvironment link 101 at 125 then sends the higher priority command, inthis example, the menu-selection command from the application profile105, to the integrated operating environment 56 which routes the commandto the active application program spreadsheet 50. The spreadsheet 50 at127 takes the appropriate action in response to that command, in thiscase, graphing the currently selected data and changing the informationpresented to the user by the display 28 accordingly.

Interface Profiles

To understand the power and flexibility of the advanced user interface100, the application profiles 105 and the user profiles 107 of theinterface profile 104 must be explained in greater detail. Both types ofprofile associate a list lists of input messages with a list ofcorresponding commands which are recognized by the appropriateapplication program, the environment link module 101, or by theintegrated operating environment 56 itself. The command can be the nameof a program, an instruction recognized by the integrated operatingenvironment or a MACRO. A MACRO is a file which contains an arbitrarilylong string of "events" and/or utilities. An "event", for the purposesfor this specification, is an input message which results from a useraction such as a single keystroke, a mouse button press, a mouse buttonrelease, a mouse double click, single menu selection, a single gestureor other "token", or a single accelerator key selection. An example of autility is the pop-up keyboard. A MACRO may also treat other previouslydefined MACROS as commands. Accelerator keys are often a feature of theintegrated operating environment 56. They specify a keystroke sequencewhich emulates a menu selection. For example, an application may specifythat the DELETE key is the same as selecting CUT from the EDIT menu andthat SHIFT/INSERT is the same as selecting PASTE from the EDIT menu.These two examples are, in fact, widely used accelerators.

Each application program may have its own application interface profile105, which contains mappings of input messages and their correspondingcommands for each input device connected to the alternate inputsubsystem 103. For example, one application profile might contain inputmessages and commands as shown in FIG. 5. The sample application profileshows input messages mapped against the corresponding command togetherwith comments for display to the user describing the user action neededto create the input message and comments describing the action thecomputer system will take upon receiving that input message when thegiven application is active.

The same input message from the AIS 103 will generally invoke adifferent MACRO on different application profiles 105. For example, acertain application profile 105 may specify that a MACRO named"Cut/Paste" should be executed in response to a Right-Arrow gesture,while a different application profile 105 may map the Right-Arrowgesture to the keystroke sequence "Form 201 (ENTER)". When the gesturerecognition unit 64 detects a right arrow gesture from the touch sensor36, it calls environment link 101 which determines the applicationprogram which owns the currently active window. The environment link 101reads the corresponding application's profile 105, merges thatinformation with the appropriate user's profile 107, choosing thecommand with higher priority, then issues the command to the activeapplication program the integrated operating environment 56 or to otherAUI modules as appropriate.

The user profile 107 contains commands which are common to all orseveral of the application programs 50-53. A separate user profile 107can be written for each system user. The advanced user interface candetermine the active user during log-on procedures. Entries in the userprofile 107 can be used to override those in the application profiles105. This situation might occur when a particular user prefers to inputdata in a different manner from that provided by the application profile105 written for a group of system users. The user profile 105 can alsobe used to provide a global default action, i.e., if a particular inputmessage was not found by PM-Link 101 in the appropriate applicationprofile 105. Where a user profile 107 is used for a global defaultaction, the application profile 105 which has a MACRO for a given inputmessage can be given a higher priority than the default action.

While the preferred embodiments of the present invention make adistinction between the application profiles 105 and the user profiles107, in other embodiments, it is possible to write separate applicationprofiles 105 for each user. Where the interface profile module 104 wasorganized in this manner, if an application profile 105 did not existfor a particular user, the environment link 101 would refer to a defaultapplication interface profile 105. The default application profile 105would be provided with the application program, the input device or bythe programmer configuring the computer system. As currently envisioned,an application profile 105 for a given application program wouldgenerally be provided with the application itself. Alternatively, allapplication profiles 105 as well as a general user profile 107 could beprovided by the system programmer. For an unsophisticated user who doesnot wish to explore the power of the advanced user interface 100, thedefault profiles will usually be adequate. However, many users will wishto tailor the advanced user interface 100 to their particular desiresand specification. Therefore, it will be necessary to either write ormodify an existing application or user profile.

The Advanced User Interface Control Panel, one of the Advanced UserInterface Utilities 109, allows the user to tailor many of the AUIfunctions to create a user unique profile in an extremely efficient anduser friendly manner. Although the advanced user interface 100 containsfairly sophisticated computer code, through the use of the ControlPanel, the user need not understand its workings. Among the functionsoffered by the AUI Control Panel is the ability to modify or createprofiles. This is accomplished by invoking the interface profilecreate/modify module of the AUI Control Panel by selecting "PROFILE"from the Control Panel menu. The Control Panel first queries the userthe name of the profile to be modified. At this point, a menu of theappropriate profile is displayed to the user, including a list ofdescriptions of user actions, e.g., circle gesture, "cut" voice command,etc., against a description of the action taken by the computer, i.e.,the MACRO, in response to the user action. If there is no application oruser profile associated with the name entered, the utility creates a newprofile and displays a blank menu. The user moves the cursor through themenu to add or modify the information in the profile, and then invokesthe record command function or the record MACRO function. For example,returning to the first item in FIG. 5, a single line of the menu for thespreadsheet application profile may read:

    ______________________________________                                        USER ACTION     COMMAND                                                       ______________________________________                                        CIRCLE gesture  Invoke the pop-up keyboard                                    on touch sensor                                                               ______________________________________                                    

If the user wants to change the input message from the circle gesture toa right hand arrow gesture, he moves the cursor to the user action fieldand selects "USER ACTION" from the menu. The user is prompted to performthe action by which he wishes to invoke the pop-up keyboard, and then isprompted to add a written description of the action to be displayed onthe menu. Similarly, if the user wishes to change the MACRO invoked bythe circle gesture, he moves the cursor to the MACRO field and entersthe name of the MACRO to be executed.

The user creates a new profile in a similar fashion as an existingprofile is modified. First, moving the cursor to a blank user actionfield, selecting "USER ACTION" from the menu, then when finished, movingthe cursor to the corresponding macro field, entering a MACRO name orother command, and so forth until the profile is complete. Thus, itbecomes a relatively straightforward procedure for a relativelyunsophisticated user to construct the advanced user interface profilesfor the application programs he has selected to allow him to input datainto a computer system in the most advantageous manner.

A MACRO can be created independently from the creation of a user orapplication profile. In Presentation Manager™ 56, for example, there arethree cooperating utilities which allow the user to create a MACRO,modify a MACRO and execute a MACRO.

The Macro Creation Utility (MCU) is a Presentation Manager™ programwhich can be set up as a tutorial, a prompter, or a totally invisibleutility. It can be called from the standard program loader or via adesignated gesture or other input signal. Its job is to record theuser's sequence of commands for future playback. The user merely startsthe MCU, performs the desired sequence of operations, then stops theMCU. The MCU then prompts the user for required information such as thename of the MACRO. Once the macro is named, the user can control the useof that MACRO by referring to it in any of his application profiles orin his user profile.

The Macro Modify Utility (MMU) is a Presentation Manager™ program whichallows the user to single step through a previously created macro. MMUdisplays an English language description of each step in the macrobefore it executes. At any point in the playback process, the user canmodify one or more commands in the macro. The modification may beperformed by editing the contents of the English language window (forexperienced macro writers) or by turning on the "record" function tooverwrite previously stored commands. The MMU also allows the user toeasily test, debug and modify MACROs. It also lets the user build MACROsby modifying existing MACROs which may already do most of what the userneeds.

The Macro Execution Utility (MEU) is a Presentation Manager™ programwhich plays back the macro at full speed. The MEU could be called by theenvironment link 101 whenever the user issues the appropriate gesture.For example, if a left arrow gesture is recognized, the environment link101 looks in the current User's Profile to determine if that gesture is"globally" inhibited. If not, the environment link 101 then determinesthe name of the active application and reads the correspondingApplication Profile to determine what (if any) MACRO is linked to theleft arrow gesture, for this application. The environment link 101 thencalls the MEU, if necessary.

One example of a default user profile is described with reference to atouch input device substituting for a mouse pointing device. As manyprograms recognize mouse input, a great many application programs couldbe operated using this default profile.

In this example, the advanced user interface can be set in one of aplurality of modes which determine what mouse equivalent commands arefound in the user profile as a result of stylus or finger actions. Whenthe AUI is in mode one, no mouse button messages are generated bytouchdown, mouse move commands are generated while the finger or stylusis moving in the touch panel. Upon lift-off of the stylus, a buttondown,buttonup command sequence is sent to the application program. Mode oneallows accurate positioning of the cursor before action is taken by theactive application. When the advanced user interface is in mode two, nomouse button commands are generated by touchdown, mouse move messagesare generated by moving the stylus, and a mouse double click isgenerated upon lift-off. When the AUI is in mode three, a mouse buttondown command is generated on contact and a mouse button up command onlift-off of the stylus. Mode three is useful for dragging screen objects(icons, etc), sizing windows and marking areas in a window's client area(e.g., marking text which is to be moved). These modes may be selectedmanually by touching the touch panel repeatedly at an icon whichindicates which mode the AUI is in, or automatically by the applicationprofile.

Second Embodiment

FIG. 6 shows another embodiment of the Advanced User Interface, wherePresentation Manager™ 156 is used as the integrated operatingenvironment, and OS/2 154 is the operating system which runs thecomputer system. Because of the peculiarities of OS/2™ 154 andPresentation Manager™ 156, a slightly different arrangement of theAdvanced User Interface 200 is necessary. OS/2™ 154 has a single queuemodule which timestamps all input and requires that all input be queuedwithin this module in the order in which it was transmitted to thecomputer system. Therefore, all of the alternate input devices must gothrough OS/2™ 154 before being handled by the environment link modulePM-Link 201. In addition, the present implementation of the single queuefeature allows only keyboard and mouse input, and will not handle inputfrom a touch input device, or other alternate input devices.

These constraints lead to the advanced user interface design shown inFIG. 6. All input devices keyboard 130, device C 132, device D 134,device E 136 and device F 138, go through OS/2™ 154 to be added to thesingle queue of OS/2™ 154. The keyboard 130 goes directly through astandard device driver 140 to OS/2™ 154. Device C 132, device D 134,device E 135 and device F 138, go through device-specific code 141, 142,143 and 144, respectively, to the alternative input subsystem 203,before being passed to the single queue of OS/2™ 154. Since the singlequeue will only handle keyboard or mouse input, the alternative inputsubsystem 203 converts the input signals from device C 132, device 134,device 136 and device 138 into mouse or keyboard input messages. Inaddition, since OS/2™ 154 will not handle touch, voice or image "tokens"as input messages, all such inputs are passed through as a series ofmouse move points or other "events". As in the previous embodiment, theAIS 203 keeps track of the time which each input signal was received aswell as a record of the true input signal. OS/2™ 154 then passes theinput messages to Presentation Manager™ 156 which keeps track of theactive application program.

In FIG. 6, PM-Link 201, the environmental link module of the AdvancedUser Interface 200, acts as a filter between Presentation Manager™ 156and the application programs, Application E 150, Application F 151,Application G 152 and Application H 153. After receiving an inputmessage from Presentation Manager™ 156 together with information aboutwhich is the active application program, the PM-Link 201 refers back tothe alternate input subsystem 203 to determine whether the "keyboard" or"mouse" message it received is in fact a keyboard or a mouse message, orrather a touch, voice, image, or other message. The PM-Link 201 thenpasses the true message to the appropriate application program. Theremaining modules of the advanced user interface 200 are not calledunless the active application program indicates that the true inputmessage is not understood. If the active application indicates that thetrue input message is not understood, PM-Link 201 then refers to theinterface profiles module 204 to determine the correct command. If thetrue input message indicates that it may be a gesture, character, voiceor image "token", the PM-Link 201 sends the input message to theappropriate recognition unit to identify the token, and then searchesthe interface profiles 204 to determine the correct command.

The flow diagram in FIG. 7 depicts the operation of the computer systemin FIG. 6 in response to a voice input signal. The active applicationprogram is the spreadsheet 150, and presented to the user on the display28 is an array of selected data concurrently with a member of possiblefunctions available to the user. Rather than using the keyboard 130, theuser elects to input data using another input device, the voice sensor136. In response to the user vocalizing the word "GRAPH", the voicesensor 136 generates electrical signals and thereby invokes theappropriate device specific code 144 is invoked at 221. The devicespecific code at 223 then sends a series of input signals to the AIS 203corresponding to the word "GRAPH". The AIS 203 translates the inputsignals as a series of keyboard or mouse input messages to be added tothe single queue of OS/2™ 154 at 225. After timestamping the stream ofinput messages, OS/2™ passes them to Presentation Manager™ 156 whichkeeps track of the focus of the user and passes the input messages tothe active application program, the spreadsheet 150. The PM-Link 201intercepts the message to the active application program at 227 andrefers back to the AIS 203, to determine that the true input messagesare a set of voice input messages at 229 and sends the true inputmessages, the voice input, to the spreadsheet 150 at 231.

After the PM-Link 201 transmits the true voice input message, theapplication program will respond with its own message at 233 and one oftwo things may occur. If the application does not understand the type ofinput message sent, it returns a "R0" message to the PM-Link 201 at 233,indicating that it does not understand the input message. The PM-Link201, in response to an R0 message, would normally go to the interfaceprofiles 204 to determine the corresponding command in the appropriateapplication profiles 205 at 241 and user profile 207 at 243. However,since the input messages indicate that they are a voice message at 237,the PM-Link 201 first sends the input messages to the voice recognitionunit 215 to receive the correct voice input token for the word "GRAPH".After receiving this token at 239, the PM-Link 201 first looks in theapplication profile 205 of the spreadsheet 150 at 241 to find thecorresponding command to the input message which will be recognized bythe spreadsheet 150, i.e., the menu selection --graph--. Then thePM-Link 201 refers to the user profile 207 of the current user at 243.The PM-Link 201 then determines which profile has the higher prioritycommand for the spreadsheet 150 at 243, and then sends the higherpriority command to spreadsheet 150 at 247. The spreadsheet then takesthe appropriate action at 249, in this case, graphing the currentlyselected data and changing the information presented to the user by thedisplay 28.

If, on the other hand, the spreadsheet 150 can decipher, hand or isotherwise aware of voice input, an "R1" message is sent to the PM-Link201 at 233. The PM-Link 201 would then take no further action for thecurrent message. While the spreadsheet 150 may be aware of voice input,it may also elect to ignore the voice input in certain circumstances, orperiods during the application program. Alternatively, the awareapplication 150 may understand some voice input, but not others, andsend an R0 message back to the PM-Link 201 at 233 whenever it does notunderstand a particular voice input message. For those voice inputmessages for which it received an "R0" message, the PM-Link 201 wouldconsult the voice recognition unit 215 and the interface profiles 204 asdetailed above.

In the embodiment of FIG. 6, the character recognition 211 and thegesture recognition unit 213 are part of the Advanced User Interface200. When PM-Link 201 receives an input message which might be acharacter or gesture, it sends the set of points to the characterrecognition unit 211. If the character recognition unit 211 recognizesthe set of points as an alphanumeric character, that character isidentified and sent back to PM-Link 201 for further processing in theactive application program. If the set of points is not recognized as acharacter, the set of points is sent to the gesture recognition unit 213to determine whether the set of points is recognized as a gesture. If itis recognized as a gesture, PM-Link 201 sends the gesture to theinterface profiles 204 to find the set of instructions to send to theactive application program from the appropriate application profile 205or user profile 207. If it is not recognized as either a character or agesture, the PM-Link 201 decides that the input message was probablyintended as a mouse message and generates a "mouse buttondown, mousebuttonup" message at the last point in the set of points.

As an alternative to the default user profile described above, wherevarious modes are used to determine what mouse equivalent commands aregenerated by user actions on the touch panel, the present invention caninclude a computer module within PM-Link 201 to differentiate betweentouch input which emulates a mouse buttondown event, mouse move event ora mouse buttonup event, and that touch input which is to be considered agesture or character. This is done by using a time delay todifferentiate between the two types of touch input. The Advanced UserInterface 200 allows the user to touch the screen and move to thedesired position before the application is notified of the event byplacing all the points received from the touch sensor in a stroke bufferwhich either stylus liftoff or the stylus has delayed at a particularpoint for the set time delay. Once the user reaches the desired positionand stops moving for the set time delay, the application program willreceive the event through PM-Link 201 as a mouse event. For example, ifthe user stops moving for 200 milliseconds, a mouse buttondown - mousebuttonup event at the desired position is communicated to theapplication program. However, if the user starts to move again beforethe end of the set period of time, the event is not generated until theuser once again stops and pauses for the period of time delays. The timedelay is typically defined in milliseconds and may be varied dependingupon the area of the screen with which the user is interacting. The timedelay may be specified by the user in the user profile 207 or may bealtered by the application program which is utilizing the touch input.

If, on the other hand, the user does not stop at a particular point forthe specified time delay period and instead lifts the stylus off thetouch screen, the Advanced User Interface 200 selects the set of inputpoints as a candidate for character or gesture recognition. In otherwords, the time delay provides a window in time in which gestures can bemade. During this time, the Advanced User Interface 200 will only allowgestures to be made. If the user waits for the time delay period beforelifting off, the points collected will not be candidates for a gesture,but a mouse event. If the user lifts off before the time delay periodexpires, an attempt will be made to recognize the set of input points inthe character recognition unit 211 or gesture recognition unit 213 as agesture or character. If it is not recognized, however, the normal mouseemulation sequence will be generated: mouse buttondown, mouse buttonupat the last input point in the set. If it is recognized, the PM-Link 201refers to the interface profiles 204 to send the appropriate commands tothe active application program.

The recognition of a circle gesture drawn by a user on the face of thetouch sensor 134 is described below with reference to FIG. 8.

A user having determined that he wishes to invoke the pop-up keyboardutility in the AUI utilities module 209, draws a circle on the face ofthe touch sensor 134 and lifts the stylus off before the end of the settime delay. The touch sensor 134 generates a series of interrupts to thedevice specific code 142 at 261 which passes a set of input signals tothe AIS 203 corresponding to the set of points in the circle at 263. TheAIS 203 translates the input signals to mouse input messages to behandled by the single queue in OS/2™ 154 at 265. The input messages aresent to the active application program, the spreadsheet 150, byPresentation Manager™ 156, but are intercepted first by the PM-Link 201at 267. The PM-Link 201 then queries the AIS 203 and determines that the"mouse" input messages are actually touch input at 269.

When PM-Link 201 discovers the first touch input point, usually a touchdown on the sensor 134, it refers to the user profile 207 for the timedelay period selected by the user at 271. At 273, the PM-Link 201collects the touch input points in its stroke buffer until the stylusliftoff event is detected. As the PM-Link 201 collects the touch inputpoints, each time the user pauses on the touch sensor within the stroke,initiating a "period of stability" at 275, the PM-Link 201 starts timingthe period of stability to determine whether the time delay period hasexpired at 277. If the user moves before the time delay period hasexpired, the PM-Link 201 continues collecting the touch input points intee stroke at 273. If, however, the period of stability exceeds the timedelay, a mouse buttondown, mouse buttonup command is generated at 281.In this example, the user does not pause before the stylus is lifted,and therefore, the stroke is sent to the character recognition unit 211and gesture recognition unit 213 for processing at 279. If the touchinput stroke had not been recognized as either a gesture or character, amouse buttondown, mouse buttonup command would be generated at 281.

After being recognized by the gesture recognition unit 213 as a circlegesture, the PM-Link 201 passes the circle gesture to the spreadsheet150. The spreadsheet returns an "R0" message at 285, indicating that thecircle gesture was not understood. Since the circle gesture was notunderstood at 287, the PM-Link 201 refers to the application profile 205for the spreadsheet 150 at 289 and finds no corresponding command forthe circle gesture. The PM-Link 201 then refers to the user profile 207at 291 which contains the corresponding command "invoke the pop-upkeyboard". As the command from the user profile 207 has the highestpriority at 293, the PM-Link 201 sends this message to the AUI utilitiesmodule 209 at 295 and the pop-up keyboard is presented to the user onthe display.

While the present invention has been described with reference tospecific embodiments, it will be understood by those skilled in the artthat changes in form and detail may be made without departing from thespirit and scope of the invention. For example, while gesture andcharacter recognition has discussed in terms of a touch input device,another pointing input device such as a mouse or a graphic tablet can beutilized to send gestures or characters to the Advanced User Interface.In addition, while specific mention has been made of particularapplication programs, integrated operating environments and inputdevices, the flexibility of the Advanced User Interface can accommodateboth further advances as well as the existing implementations of theseprograms and devices. The embodiments presented are for purposes ofexample and illustration only and are not to be taken to limit the scopeof the invention narrower than the scope of the appended claims.

We claim:
 1. An advanced user interface, for use with an integratedoperating environment and an operating system capable of running aplurality of application programs simultaneously on a computer system,only one of which is active at a given time, the computer system havinga central processor, a random access memory, a display and at least oneinput device which transmits input signals to the advanced userinterface in response to actions performed by a user,comprising:interface profiles which contain mappings of input messagesrepresentative of the input signals against corresponding commandsuseable by at least one of the plurality of application programs; and,an environment link module coupled to the interface profiles, theintegrated operating environment and the operating system, theenvironment link module receiving the input messages, determining whichof the plurality of application programs was active at the time theinput signals were transmitted by the input device, matching the inputmessages against the corresponding commands contained in the interfaceprofiles according to the application program which was active at thetime the input signals were transmitted by the input device, and sendingthe corresponding commands to an appropriate code module; the computersystem performing a given action in response to user actions differentfrom those user actions for which the active application program wasoriginally written without change to the active application program. 2.The advanced user interface as recited in claim 1 which furthercomprises an alternative input subsystem coupled to the input device andthe environment link which translates the input signals to the inputmessages.
 3. The advanced user interface as recited in claim 1, whereinthe interface profiles comprise a plurality of application profiles eachof which corresponds to one of the plurality of application programs,the application profiles containing mappings of input messages againstcorresponding command at least some of which are useable by itscorresponding application program.
 4. The advanced user interface asrecited in claim 3, wherein the interface profiles further comprise auser profile which contains mappings of input messages againstcorresponding commands, the user profile useable with more than one ofthe plurality of application programs.
 5. The advanced user interface asrecited in claim 4, wherein at least one of the correspondinginstructions contained in the user profile overrides at least one of thecorresponding commands contained in the plurality of applicationprofiles.
 6. The advanced user interface as recited in claim 4, whereinat least one of the corresponding commands contained in the user profileis a default instruction which is sent only when the application profileof the active application program has no corresponding commands to theinput message.
 7. The advanced user interface as recited in claim 4,wherein the interface profiles further comprise a user profile for eachuser authorized to utilize the computer system.
 8. The advanced userinterface as recited in claim 2, wherein a plurality of input devicesare coupled to the alternative input subsystem module.
 9. The advanceduser interface as recited in claim 8, wherein the interface profilescontain mappings of input messages from a different input device fromthe input devices for which at least one of the plurality of applicationprograms were originally written.
 10. The advanced user interface asrecited in claim 9 wherein the interface profiles contain mappings oftouch input messages against corresponding mouse commands.
 11. Theadvanced user interface as recited in claim 10 wherein the interfaceprofiles also contain mappings of touch input gesture tokensrepresenting a series of points drawn by a user on an input surfacedetected by a touch input device against corresponding commands.
 12. Anadvanced user interface, for use with an integrated operatingenvironment and an operating system capable of running a plurality ofapplication programs simultaneously on a computer system, only one ofwhich is active at a given time, the computer system having a centralprocessor, a random access memory, a display and at least one inputdevice which transmits input signals to the advanced user interface inresponse to actions performed by a user, comprising:an alternative inputsubsystem module coupled to the input device which translates the inputsignals to input messages compatible with the integrated operatingenvironment; interface profiles which contain mappings of the inputmessages against corresponding commands useable by at least one of theplurality of application programs; and, an environment link modulecoupled to the alternative input subsystem, the interface profiles andthe integrated operating environment, the environment link modulereceiving the input messages, intercepting the compatible input messagesbefore receipt by the active application program, querying thealternative input subsystem as to whether the compatible input messagesare true input messages, determining which of the plurality ofapplication programs was active at the time the input signals weretransmitted by the input device, matching the true input messagesagainst the corresponding commands contained in the interface profilesaccording to the application program which was active at the time theinput signals were transmitted by the input device and sending thecorresponding commands to an appropriate code module; the computersystem performing a given action in response to user actions differentfrom those user actions for which the active application program wasoriginally written without change to the application program.
 13. Theadvanced user interface as recited in claim 12, wherein theenvironmental link first sends the true input messages to theapplication which was active at the time the input messages weretransmitted by the input device, and responsive to a message indicatingthat the active application did not understand the true input message,matching the true input messages against the corresponding commandscontained in the interface profiles according to the application programwhich was active at the time the input signals were transmitted by theinput device and sending the corresponding commands to an appropriatecode module.
 14. A computer system having a central processor and arandom access memory having an advanced user interface, an integratedoperating environment and an operating system resident in the memorycapable of running a plurality of application programs simultaneously,only one of which is active at a given time, comprising:a plurality ofinput devices which transmit input signals to the advanced userinterface in response to actions performed by the user; translatingmeans coupled to the plurality of input devices to translate the inputsignals to input messages; interface profiles which contain mappings ofthe input messages against corresponding commands useable by at leastone of the plurality of application programs; an environment link modulecoupled to the translating means input subsystem, the interface profilesand the integrated operating environment, the environment link modulereceiving the input messages, determining which of the applicationprograms wa active at the time the input signals were transmitted by theinput devices, matching the input messages against the correspondingcommands contained in the interface profiles for the application programwhich was active at the time the input signals were transmitted by oneof the plurality of input devices, and sending the correspondingcommands to an appropriate code module; and, a display that presentsinformation concerning the status, results, choices and other itemsassociated with the plurality of application programs; the computersystem performing a given action in response to user action differentfrom those user actions for which the active application program wasoriginally written without change to the active application program. 15.The computer system as recited in claim 14, which further comprises asecond plurality of input devices coupled to the integrated operatingenvironment through standard device drivers.
 16. The computer system asrecited in claim 15, wherein at least one of the second plurality ofinput devices is also coupled to the translating means.
 17. The computersystem as recited in claim 14, which further comprises a gesturerecognition unit which interprets the input signals transmitted by atouch input device in response to a user drawing symbols on a surfacedetected by the touch input device.
 18. The computer system as recitedin claim 14, which further comprises a character recognition unit whichinterprets the input signals transmitted by a touch input device inresponse to a user writing alphanumeric characters on a surface detectedby the touch input device.
 19. The computer system as recited in claim14, wherein input messages of a touch input device are mapped againstcorresponding mouse commands for at least one of the plurality ofapplication programs.
 20. The computer system as recited in claim 14,wherein the interface profiles comprise a plurality of applicationprofiles each of which is for one of the plurality of applicationprograms containing mappings of input messages against correspondingcommands at least some of which are useable by its correspondingapplication program.
 21. The computer system as recited in claim 20,wherein the interface profiles further comprise a user profile whichcontains mappings of input messages against corresponding commands, theuser profile useable with more than one of the plurality of applicationprograms.
 22. The computer system as recited in claim 21, wherein atleast one of the corresponding commands contained in the user profileoverrides at least one of the corresponding commands contained in theplurality of application profiles.
 23. The computer system as recited inclaim 21, wherein at least one of the corresponding commands containedin the user profile is a default command which is sent only when theapplication profile of the active application program has nocorresponding command to the input message.
 24. The computer system asrecited in claim 21, wherein the interface profiles further comprise auser profile for each user authorized to utilize the computer system.25. The computer system as recited in claim 14, which further comprisesa touch input device disposed over the viewing surface of the displaydevice, and wherein the advanced user interface further comprises apop-up keyboard module which displays an image of a keyboard on thedisplay device and in response to a selection of a key in the keyboardtransmits a character string to the an appropriate computer module inthe random access memory.
 26. The computer system as recited in claim14, wherein the advances user interface further comprises an imagemagnifying utility which magnifies a selected portion of an imagedisplayed on the display device so that a more precise pointingoperation can be accomplished.
 27. The computer system as recited inclaim 14, wherein the interface profiles contain mappings of inputmessages from a different input device form the input devices for whichat least one of the plurality of application programs was originallywritten.
 28. The computer system as recited in claim 14, wherein theadvanced user interface further comprises an advanced user interfacecontrol panel to allow a user to create and modify the advanced userinterface in a user friendly manner.
 29. A computer system having acentral processor and a random access memory having an operating system,an advanced user interface and an integrated operating environment andan operating system resident in the memory capable of running aplurality of application programs simultaneously, only one of which isactive at a given time, comprising:a plurality of input devices whichtransmit input signals to the advanced user interface in response toactions performed by the user; an alternative input subsystem modulecoupled to the input device which translates the input signals to inputmessages compatible with the integrated operating environment and theoperating system; interface profiles which contain mappings of the inputmessages against corresponding commands useable by at least one of theplurality of application programs; and, an environment link modulecoupled to the alternative input subsystem, the interface profiles andthe integrated operating environment, the environment link modulereceiving the input messages, intercepting the compatible input messagesbefore receipt by the active application program, querying thealternative input subsystem as to whether the compatible input messagesare true input messages, determining which of the plurality ofapplication programs was active at the time the input signals weretransmitted by the input device, matching the true input messagesagainst the corresponding commands contained in the interface profilesaccording to the application program which was active at the time theinput signals were transmitted by the input device and sending thecorresponding commands to an appropriate code module; and, a displaythat presents information concerning the status, results, choices andother items associated with the plurality of application programs; thecomputer system performing a given action in response to user actionsdifferent from those user actions for which the active applicationprogram was originally written without change to the applicationprogram.
 30. The computer system as recited in claim 29, wherein theenvironmental link sends the true input messages to the applicationwhich was active at the time the input messages were transmitted by theinput device and only responsive to a message indicating that the activeapplication did not understand the true input message, matching the trueinput messages against the corresponding commands contained in theinterface profiles according to the application program which was activeat the time the input signals were transmitted by the input device andsending the corresponding commands to an appropriate code module.
 31. Aset of interface profiles for use with an advanced user interface incombination with an integrated operating environment and an operatingsystem capable of running a plurality of application programssimultaneously on a computer system only one of which is active at agiven time, the set of interface profiles comprising mappings of inputmessages representative of the input signals of an input device coupledto the advanced user interface mapped against corresponding commandsuseable by at least one of the plurality of application programs whenactive, the advanced user interface referring to the interface profilesresponsive to a determination of which of the plurality of applicationprograms was active at the time an input signal was sent so that thecomputer system performs a given action in response to user actionsdifferent from user actions for which at least one of the applicationprogram was originally written without change to the application. 32.The set of interface profiles as recited in claim 31, which furthercomprise a plurality of application profiles each of which correspondsto one of the plurality of application programs containing mappings ofinput messages against corresponding commands at least some of which areuseable by its corresponding application program.
 33. The set ofinterface profiles as recited in claim 32, which further comprise a userprofile which contains mappings of input messages against correspondingcommands, the user profile useable with more than one of the pluralityof application programs.
 34. The set of interface profiles as recited inclaim 33, wherein at least one of the corresponding commands containedin the user profile overrides at least one of the corresponding commandscontained in the plurality of application profiles.
 35. The set ofinterface profiles as recited in claim 34, wherein at least one of thecorresponding commands contained in the user profile is a defaultinstruction which is sent only when the application profile of theactive application program has no corresponding command to the inputmessage.
 36. The set of interface profiles as recited in claim 34,wherein the interface profiles further comprise a user profile for eachuser authorized to utilize the computer system.
 37. The set of interfaceprofiles as recited in claim 31, wherein the interface profiles containmappings of touch input messages against corresponding mouse commands.38. The set of interface profiles as recited in claim 31, wherein theinterface profiles also contain mappings of touch input gesture tokensagainst corresponding commands, the touch input gesture tokensrepresenting a series of points drawn by a user on an input surfacedetected by a touch input device.
 39. A method of operating a computersystem with an integrated operating environment and an operating systemcapable of running a plurality of application programs simultaneouslyonly one of which is active at a given time by the use of an advanceduser interface, the computer system having a central processor, a randomaccess memory, a display and at least one input device which transmitsinput signals to the advanced user interface, comprising the stepsof:determining which one of the application programs was active at thetime the input signals were transmitted by the input device; finding acorresponding command to the input message representative of the inputsignals in a set of interface profiles for the active applicationprogram, the interface profiles comprising mappings of the inputmessages against corresponding commands; and, sending the correspondingcommand to an appropriate code module; the computer system performing agiven action in response to user actions different from those useractions for which the active application program was originally writtenwithout change to the active application program.
 40. The method ofoperating a computer system as recited in claim 39 which furthercomprises the steps of:receiving the input signals transmitted from theinput device in response to actions performed by a user; and,translating the input signals into input messages.
 41. The method ofoperating a computer system as recited in claim 39 wherein the inputdevice is a touch input device, and the interface profiles containmappings of touch input messages against corresponding mouse commands.42. The method of operating a computer system as recited in claim 41,wherein the interface profiles also contain mappings of touch inputgesture tokens against corresponding commands, the touch input gesturetokens representing a series of points drawn by a user on an inputsurface detected by a touch input device.
 43. The method of operating acomputer system as recited in claim 39, wherein the finding stepcomprises the steps of:finding a corresponding command to the inputmessages in an application profile for the active application; finding acorresponding command to the input messages in a user profile for theuser operating the computer system; and, determining which correspondingcommand has higher priority.
 44. The method of operating a computersystem as recited in claim 39, wherein the corresponding command is sentto the active application program.
 45. The method of operating acomputer system as recited in claim 39, wherein the correspondingcommand is sent to a utilities module of the advanced user interface.46. A method of operating a computer system with an integrated operatingenvironment and an operating system capable of running a plurality ofapplication programs simultaneously only one of which is active at agiven time by the use of an advanced user interface, comprising thesteps of:receiving input signals transmitted from at least one inputdevice in response to actions performed by a user; translating the inputsignals into an input message compatible with the integrated operatingenvironment; intercepting the compatible input message before receipt byan active application program; determining whether the compatible inputmessage is a true input message; determining which one of theapplication programs was active at the time the input signals weretransmitted; finding a corresponding command to the true input messagein a set of interface profiles for the active application program, theinterface profiles comprising mappings of the true input messagesagainst corresponding commands; and, sending the corresponding commandto an appropriate code module; the computer system performing a givenaction in response to user actions different from those user actions forwhich the active application program was originally written withoutchange to the active application program.
 47. The method of operating acomputer system as recited in claim 46 which further comprises the stepof sending the true input message to the application which was active atthe time the input signal were received and wherein only responsive to amessage that the active application did not understand the true inputmessage performing the finding and sending steps.